Click any tag below to further narrow down your results
Links
JOLT is a training method where one model plays both teacher and student, with the teacher trained to give feedback the student can actually learn from. The authors report gains in accuracy and token efficiency over GRPO on math, coding, and agentic tasks.
- Outcome-reward RL gives sparse feedback, and on long agentic tasks with rare successes that makes learning slow and sample-hungry.
- Privileged-information teachers (like a model that sees the verified solution) can be good at the task but poor at teaching it, since their guidance may exceed what the student can absorb.
- JOLT trains the teacher to be "reward-consistent," meaning its guidance pushes the student in the same direction as RL on the student's own reward. Reported gains over GRPO include +4.2pp on MATH, +3.0pp on LCBv6, and +9.9pp on AppWorld.
Periodic trained an AI model called Neon that outperforms frontier models like GPT-6 at analyzing X-ray diffraction data—a task materials scientists spend hours on—using less compute and lower cost. The model learned from experimental lab data through reinforcement learning with expert judgment, achieving a 55% success rate on their hardest internal benchmark.
- Neon reached 55.3% success on FrontierXRD (134 complex samples), a 20x jump from the base model's 2.7%, while costing less per analysis than GPT-6 Astra or Claude Fable 5.1
- Periodic built a custom scientific harness that achieved 3.8x higher success rates than Claude Code with standard tools, showing that model capability depends heavily on available databases and software
- The company used an LLM-judge ensemble calibrated to human expert ratings (74.6% agreement with humans, 84% with consensus) to generate training signals for reinforcement learning on tasks without ground-truth answers
The author ported a Python simulation of weakly electric fish to C and optimized it to run at one million steps per second—a 250x speedup over the original. This enabled training RL policies for the fish in under 15 seconds while maintaining biological accuracy, demonstrating how performance engineering unlocks rapid experimentation in complex simulations.
- Ported a weakly electric fish RL simulation from Python to C, boosting speed from 4,300 to 1.85M steps/sec (250x), enabling full policy training in under 10-15 seconds on a single GPU.
- Exploiting the biological detail that sensory receptors have limited range (4-10cm vs 100cm) let them skip most fish-to-fish field calculations, fixing the main bottleneck.
- Switching from double- to single-precision floats alone gave a 30% speedup.
- Shows that fast iteration—borrowed from autonomous driving RL practices—can work even in messy, physics-heavy biological simulations, and the whole thing is only 1,200 lines of open-sourced C code.
+ weakly electric fish
reinforcement learning
+ performance engineering
+ simulation
+ optimization
+ biology
This post highlights the first book that pulls together language modeling, inference optimization, reinforcement learning, system scaling, agentic AI, retrieval-augmented generation, memory, environments, and benchmarks in one volume. It then points you to paperswithcode.co’s “most cited” list and recommends reading the top ten papers, coding them, and writing about your findings.
- A single book reportedly covers language modeling, inference optimization, RL, system scaling, agentic AI, RAG, memory, and benchmarks together—rare breadth even after five years of rapid AI progress.
- Recommended self-study path: go to paperswithcode.co's "most cited" list and work through the top ten papers.
- Suggested pace is one to two papers per week, each time reading, breaking down the math, building a toy implementation, and writing up findings.
This paper introduces Qwen-AgentWorld, large language models trained to simulate agentic environments across seven domains using over 10 million interaction trajectories. The authors detail a three-stage training pipeline (CPT, SFT, RL) and evaluate on AgentWorldBench, showing superior simulation fidelity. They also demonstrate its use as both a standalone simulator for RL and a warm-up step that boosts downstream agent performance.
- Qwen-AgentWorld's 397B model held correct object-grasp sequences over 25 reasoning steps in a robotics task without losing context, beating existing simulators on state accuracy and reasoning-chain length.
- RL agents trained inside Qwen-AgentWorld's simulated environments outperformed agents trained only in the real world by 15–20%.
- Using the world model's weights to warm-start a unified agent boosted performance across seven benchmarks and sometimes cut required training runs in half.
+ qwen agentworld
+ world models
reinforcement learning
+ robotics
+ simulation
+ agents
+ reasoning
This article evaluates how new frontier models—Claude Fable 5, Opus 4.8, and GPT-5.5—perform on the long-horizon FrogsGame post-training task. Fable 5 stands out by programmatically generating high-quality supervision traces via a backtracking algorithm, combining them with RL, and reliably improving the base model where others still struggle with noisy data and evals.
- Claude Fable 5 won by generating correct solution traces via a backtracking algorithm rather than fine-tuning on the base model's noisy outputs, then combining that SFT data with RL within a 20-hour budget.
- Opus 4.8 tripled its predecessor's performance (30.9% pass@4) and GPT-5.5 reached 9.7%, but both still relied on the base model's flawed self-generated data rather than an independent correct-answer source.
- Fable 5's self-assessment was far better calibrated (1.2x overconfidence vs. Opus 4.8's 4.9x) and it used nearly the full time budget while producing 3x more diverse training data.
- Evaluation itself remains a weak point across all models—small sample sizes and noisy metrics make it hard to reliably judge real progress on the task.
Jane Street builds its trading systems in OCaml, relying on the type system to catch errors at compile time and avoid run-time crashes. They snapshot every successful build to train an in-house AI model, rewarding only code changes that compile and pass tests.
- Jane Street runs its trading systems almost entirely on OCaml, betting on compile-time type checking (e.g., mandatory Option handling) to prevent costly runtime bugs.
- They rejected off-the-shelf tools like Copilot and Claude in favor of an in-house AI model trained on their own workspace snapshots.
- Developer workspaces are snapshotted every 20 seconds, and reinforcement learning rewards only edits that compile and pass tests.
- This snapshot-trained system underpins infrastructure supporting roughly $40 billion in annual trading volume.
Satya Nadella argues that companies should build a continuous learning loop combining human capital—expertise, judgment, relationships—with token capital—their own AI models—to create compounding institutional IP. He warns against a few dominant AI systems capturing all value and calls for private evals, reinforcement learning, and architectures that let firms swap general models without losing proprietary expertise.
- Companies should pair "token capital" (their own AI models) with human capital in a feedback loop where each amplifies the other, rather than treating AI as a static tool
- Firms need private evals and reinforcement learning based on internal outcomes (sales lift, error reduction) instead of public benchmarks, so they can swap general models without losing proprietary expertise
- If AI value concentrates in a few dominant providers, it risks hollowing out industries the way early globalization did
- The real competitive battle is building a compounding, firm-specific learning loop rather than owning the best large model
The article explores the concept that AI advancements follow a predictable pattern, which the author refers to as “straight lines on graphs.” It discusses the uneven capabilities of AI across different tasks while suggesting that the rate of improvement remains consistent. The author also speculates on the impact of reinforcement learning and compute resources on future AI development.
- AI task-horizon length (how long a task the AI can complete) doubles every 3–7 months, and this rate holds fairly steady even though absolute capability varies wildly across domains.
- The apparent RL-driven acceleration in benchmarks may be an illusion caused by measured tasks overlapping with what labs specifically post-trained on, not genuine broad capability gains.
- Progress splits into general pre-training gains (lift everything) versus targeted post-training gains (lift specific benchmarks labs choose to optimize for cost/PR reasons), and that targeting can shift over time.
- Even modeled compute slowdowns are unlikely to meaningfully delay key AI milestones, since current capability growth is fast enough to hit them first.
The article discusses the new version of Claude's constitution, which outlines explicit values for AI behavior. It explains how Constitutional AI improves upon traditional human feedback by using AI-generated principles to ensure safer and more transparent model outputs. The principles aim to address ethical concerns while allowing for continuous improvement.
- Constitutional AI replaces most human feedback with AI self-critique against a written constitution, then AI-generated reinforcement learning for harmlessness, cutting the need for people to review toxic content.
- Claude trained this way got better at handling adversarial prompts without becoming less helpful, and the approach makes the values steering the model's outputs more transparent.
- The constitution draws on sources like the UN Declaration of Human Rights and other labs' safety practices, but Anthropic admits it's still skewed toward Western viewpoints and remains a work in progress shaped by trial and error.